Notation

Data on a variable X are a sequence of n observations, indexed by i: \{x_i: 1, \dots, n \}.

Summation

The summation operator adds a sequence of numbers over an index:

\sum_{i=1}^{n} x_i \equiv x_1 + x_2 + \dots + x_n.

The sum of x_i from 1 to n.

Summation

The summation operator adds a sequence of numbers over an index:

\sum_{i=1}^{n} x_i \equiv x_1 + x_2 + \dots + x_n.

The sum of x_i from 1 to n.

Summation: Rule 01

For any constant c,

\sum_{i=1}^{n} c = nc.

Summation: Rule 02

For any constant c, \sum_{i=1}^{n} cx_i = c \sum_{i=1}^{n} x_i.

Summation: Rule 03

If \{(x_i, y_i): 1, \dots, n \} is a set of n pairs, and a and b are constants, then

\sum_{i=1}^{n} (ax_i + by_i) = a \sum_{i=1}^{n} x_i + b \sum_{i=1}^{n} y_i

Summation: Caution 01

The sum of the ratios is not the ratio of the sums: {\color{#81A1C1}{\sum_{i=1}^{n} x_i / y_i}} \neq \color{#B48EAD}{\left(\sum_{i=1}^{n} x_i \right) \Bigg/ \left(\sum_{i=1}^{n} y_i \right)}

Summation: Caution 02

The sum of squares is not the square of the sums: \color{#81A1C1}{\sum_{i=1}^{n} x_i^2} \neq \color{#B48EAD}{\left(\sum_{i=1}^{n} x_i \right)^2}

Essential definitions

Experiment:

Any procedure that is infinitely repeatable and has a well-defined set of outcomes.

Essential definitions

Sample Space:

The set of potential outcomes an experiment could generate

Ex. The sum of two dice is an integer from 2 to 12.

Random variables

Notation: Capital letters for random variables (e.g., X, Y, or Z) and lowercase letters for particular outcomes (e.g., x, y, or z).

Discrete random variables

A random variable that takes a countable set of values.

Discrete Random Variables: Probabilities

We describe a discrete random variable by listing its possible values with associated probabilities.

If X takes on k possible values \{x_1, \dots, x_k\}, then the probabilities p_1, p_2, \dots, p_k are defined by p_j = P(X=x_j), \quad j = 1,2, \dots, k, where p_j \in [0,1] and p_1 + p_2 + \dots + p_k = 1.

Discrete Random Variables

Probability density function (pdf)

The pdf of X summarizes possible outcomes and associated probabilities:

f(x_j)=p_j, \quad j=1,2,\dots,k.

Discrete random variables Ex.

Basketball player goes to the foul line to shoot two free throws.

• X is the number of shots made (either 0, 1, or 2).
• The pdf of X is f(0)= 0.3, f(1) = 0.4, f(2) = 0.3.¹

Use the pdf to calculate the probability of the event that the player makes at least one shot, i.e., P(X \geq 1).

Continuous random variables

A random variable that takes any real value with zero probability.

Continuous random variables

Probability density functions also describe continuous random variables.

Difference between continuous and discrete PDFs

• Interested in the probability of events within a range of values.
• e.g. What is the probability of more than 1 inch of rain tomorrow?

Distributions

Function that represents all outcomes of a random variable and the corresponding probabilities.

Uniform distribution

The probability density function of a variable uniformly distributed between 0 and 2 is

f(x) = \begin{cases} \frac{1}{2} & \text{if } 0 \leq x \leq 2 \\ 0 & \text{otherwise} \end{cases}

Uniform distribution

By definition, the area under f(x) is equal to 1.

The shaded area illustrates the probability of the event 1 \leq X \leq 1.5.

P(1 \leq X \leq 1.5) = (1.5-1) \times0.5 = 0.25

Normal Distribution

The “bell curve”

• Symmetric: mean and median occur at the same point (i.e., no skew).
• Low-probability events in tails; high-probability events near center.

Normal Distribution

The shaded area illustrates the probability of the event -2 \leq X \leq 2.

• “Find area under curve” = use integral calculus (or, in practice, R). P(-2 \leq X \leq 2) \approx 0.95

Normal Distribution

Continuous distribution where x_i takes the value of any real number ({\mathbb{R}})

• Domain spans the entire real line
• Centered on the distribution mean \mu

Rule 1: The probability that the random variable takes a value x_i is 0 for any x_i\in {\mathbb{R}}

Rule 2: The probability that the random variable falls between [x_i,x_j] range, where x_i \neq x_j, is the area under p(x) between those two values

The area above represents p(x)=0.95. The values \{-1.96, 1.96\} represent the 95% confidence interval for \mu.

Moments

Quantitative measures used to describe the shape and characteristics of a probability distribution¹

Expected Value

Describes the central tendency of distribution in a single number.¹

Expected Value (discrete)

The expected value of a discrete random variable X is the weighted average of its k values \{x_1, \dots, x_k\} and their associated probabilities:

\begin{aligned} E(X) &= x_1 P(x_1) + x_2 P(x_2) + \dots +x_k P(x_k) \\ &= \sum_{j=1}^{k} x_jP(x_j). \end{aligned}

Expected Value Ex.

Rolling a six-sided die once can take values \{1, 2, 3, 4, 5, 6\}, each with equal probability. What is the expected value of a roll?

Expected value (continuous)

If X is a continuous random variable and f(x) is its probability density function, then the expected value of X¹ is

E(X) = \int_{-\infty}^{\infty} x f(x) dx.

Expected value: Rule 01

For any constant c, E(c) = c. Ex.

Expected value: Rule 02

For any constants a and b, E(aX + b) = aE(X) + b.

Expected value: Rule 03

If \{a_1, a_2, \dots , a_n\} are constants and \{X_1, X_2, \dots , X_n\} are random variables, then

{\scriptsize \color{#4c566a}{E(a_1 X_1 + a_2 X_2 + \dots + a_n X_n)} = \color{#81A1C1}{a_1 E(X_1) + a_2 E(X_2) + \dots + a_n E(X_n)}}

In English, the expected value of the sum = the sum of expected values.

Expected value: Rule 03

The expected value of the sum = the sum of expected values.

Ex. Suppose that a coffee shop sells X_1 small, X_2 medium, and X_3 large caffeinated beverages in a day. The quantities sold are random with expected values E(X_1) = 43, E(X_2) = 56, and E(X_3) = 21. The prices of small, medium, and large beverages are 1.75, 2.50, and 3.25 dollars. What is expected revenue?

Expected value: Caution

Previously, we found that the expected value of rolling a six-sided die is E \left(\text{Roll} \right) = 3.5.

• If we square this number, we get \left[E ( \text{Roll} ) \right]^2 = 12.25.

Is \left[E \left( \text{Roll} \right) \right]^2 the same as E \left(\text{Roll}^2 \right)?

Expected value: Caution

Except in special cases, the transformation of an expected value is note the expected value of a transformed random variable.

For some function g(\cdot), it is typically the case that

\color{#4c566a}{g \left( E(X) \right)} \neq \color{#81A1C1}{E \left( g(X) \right)}.

Variance

Random variables \color{#b48ead}{X} and \color{#81A1C1}{Y} share the same population mean, but are distributed differently.

Variance (\sigma^2)

Tells us how far X deviates from \mu, on average:

\mathop{\text{Var}}(X) \equiv E\left[ (X - \mu)^2 \right] = \sigma^2_X

Where: \mu = E(X).

Variance: Rule 01

\mathop{\text{Var}}(X) = 0 \iff X is a constant.

• A random variable that never deviates from its mean has zero variance.

Variance: Rule 02

For any constants a and b, \mathop{\text{Var}}(aX + b) = a^2\mathop{\text{Var}}(X).

Standard Deviation (\sigma)

The positive square root of the variance:

\mathop{\text{sd}}(X) = +\sqrt{\mathop{\text{Var}}(X)} = \sigma

Standardizing a random variable

When we’re working with a random variable X with an unfamiliar scale, it is useful to standardize it by defining a new variable Z:

Z \equiv \frac{X - \mu}{\sigma}

Covariance

For two random variables X and Y, the covariance is defined as the expected value (or mean) of the product of their deviations from their individual expected values:

\mathop{\text{Cov}}(X, Y) \equiv E \left[ (X - \mu_X) (Y - \mu_Y) \right] = \sigma_{XY}

Covariance: Rule 01

Statistical independence:

If X and Y are independent, then E(XY) = E(X)E(Y).

Covariance: Rule 02

For any constants a, b, c, and d,

\mathop{\text{Cov}}(aX + b, cY + d) = ac\mathop{\text{Cov}}(X, Y)

Correlation Coefficient

A problem with covariance is that it is sensitive to units of measurement.

Correlation Coefficient

Perfect positive correlation: \mathop{\text{Corr}}(X,Y) = 1.

Correlation Coefficient

Perfect negative correlation: \mathop{\text{Corr}}(X,Y) = -1.

Correlation Coefficient

Positive correlation: \mathop{\text{Corr}}(X,Y) > 0.

Correlation Coefficient

Negative correlation: \mathop{\text{Corr}}(X,Y) < 0.

Correlation Coefficient

No correlation: \mathop{\text{Corr}}(X,Y) = 0.

Variance: Rule 03

For constants a and b,

\mathop{\text{Var}} (aX + bY) = a^2 \mathop{\text{Var}}(X) + b^2 \mathop{\text{Var}}(Y) + 2ab\mathop{\text{Cov}}(X, Y).

Estimators

Why do we estimate things? Because we can’t measure everything

Estimators

Estimand:

Quantity that is to be estimated in a statistical analysis

Estimators Ex.

Suppose we want to know the average height of the population in the US

• We have a sample 1 million Americans

Properties of estimators

Imagine that we want to estimate an unknown parameter \mu, and we know the distributions of three competing estimators. Which one should we use?

Properties of estimators

Question: What properties make an estimator reliable?

Properties of estimators

Question What properties make an estimator reliable?

A01: Unbiasedness

Unbiasedness example

Is the sample mean \frac{1}{n} \sum_{i=1}^n x_i = \hat{\mu} an unbiased estimator of the population mean E(x_i) = \mu?

\begin{aligned} E\left[ \hat{\mu} \right] &= E\left[ \frac{1}{n} \sum_{i=1}^n x_i \right] \\ &=\frac{1}{n} \sum_{i=1}^nE\left[ x_i \right] \quad \big\} \quad \text{rule 3} \\ &=\frac{1}{n} \sum_{i=1}^n \mu \quad \quad \ \ \ \big\} \quad \text{by definition} \\ &= \mu \end{aligned}

Properties of estimators

Question What properties make an estimator reliable?

A02: Efficiency (low variance)

The central tendencies (means) of competing distributions are not the only things that matter. We also care about the variance of an estimator.

\mathop{\text{Var}} \left( \hat{\mu} \right) = E\left[ \left( \hat{\mu} - E\left[ \hat{\mu} \right] \right)^2 \right]

Lower variance estimators estimate closer to the mean in each sample

Properties of estimators

Question: What properties make an estimator reliable?

A02: Efficiency (low variance)

The bias-variance tradeoff

Should we be willing to take a bit of bias to reduce the variance

In economics/causal inference we emphasize unbiasedness

Unbiased estimators

In addition to the sample mean, there are several other unbiased estimators we will use often.

• Sample variance estimates variance \sigma^2.

• Sample covariance estimates covariance \sigma_{XY}.

• Sample correlation estimates the pop. correlation coefficient \rho_{XY}.

Unbiased estimators

Sample variance, S_X^2, is an unbiased estimator of the pop. variance \sigma^2

S_{X}^2 = \dfrac{1}{n-1} \sum_{i=1}^n (X_i - \bar{X})^2.

Unbiased estimators

Sample correlation r_{XY} is an unbiased estimator of the pop. correlation coefficient \rho_{XY}

r_{XY} = \dfrac{S_{XY}}{\sqrt{S_X^2} \sqrt{S_Y^2}}.

Sampling

Population:

A group of items or events we would like to know about.

Ex. Americans, games of chess, cats in Eugene, etc.

Sampling

Sample:

A survey of a subset of the population.

Ex. Respondents to a survey, random sample of econ students at the UO

Sampling distributions

Focus: Populations vs Samples

• How can we make inferences about a population based on a small sample of the population?
• How do we learn about an unknown population parameter of interest?

Sampling distributions

There are myriad ways to produce a sample,¹ but we will restrict our attention to simple random sampling, where

1. Each observation is a random variable.

2. The n random variables are independent.

Population vs. sample

Question: Why do we care about population vs. sample?

Population vs sample

Question: Why do we care about population vs. sample?

Population vs sample

Question: Why do we care about population vs. sample?

Population vs sample

Question: Why do we care about population vs. sample?

How in the world do I do that

# Set the seed
set.seed(12468)

# Set population and sample sizes
n_p <- 100
n_s <- 10

# Generate data
pop_df <- tibble(
  x = rnorm(n_p, mean = 2, sd = 20)
)

# Simulation
sim_df <- parallel::mclapply(mc.cores = 4, X = 1:1e4, FUN = function(x, size = n_s) {
  pop_df %>% 
    sample_n(size = size) %>% 
    summarize(mu_hat = mean(x))
    }) %>% do.call(rbind, .) %>% as_tibble()

# Create histogram of simulation
ggplot(data = sim_df, aes(mu_hat)) +
  geom_histogram(binwidth = 1, fill = hii, color = "white", size = 0.25, alpha = 0.6) +
  geom_vline(xintercept = m0, size = 2, color = hi) +
  scale_x_continuous(breaks = m0) +
  scale_y_continuous(expand = c(0, 0), limits = c(0, NA))+
  theme(axis.text.x = element_text(size = 20),
      axis.text.y = element_blank(),
      rect = element_blank(),
      axis.title.y = element_blank(),
      axis.title.x = element_text(size = 20, hjust = 1, color = hi),
      line = element_blank())

Population vs. sample

Question: Why do we care about population vs. sample?

Population vs. sample

Question: Why do we care about population vs. sample?

The Central Limit Theorem

Theorem

Let x_1, x_2, \dots, x_n be a random sample from a population with mean E\left[ X \right] = \mu and variance \text{Var}\left( X \right) = \sigma^2 < \infty, let \bar{X} be the sample mean. Then, as n\rightarrow \infty, the function \frac{\sqrt{n}\left(\bar{X}-\mu\right)}{S_x} converges to a Normal Distribution with mean 0 and variance 1.

The Central Limit Theorem

Some interesting YouTube links:

• A more in depth explanation + visualization

• What is so special about the normal distribution?

Data

There are two broad types of data

1. Experimental data

Data generated in controlled, laboratory settings¹

Data

There are two broad types of data

1. Experimental data

2. Observational data

Data generated in non-experimental settings

Data types: Cross sectional

Sample of individuals from a population at a point in time

Data types: Time series

Observations of variables over time

Data types: Pooled cross sectional

Cross sections from different points in time

Data types: Panel data

Time series for each cross sectional unit

Data types: Messy data

Analysis ready dataset are rare. Most data are messy

Data wrangling is a non-trivial part of an economist or data scientist/analyst’s job

Table of Contents

1. Notation
2. Basic probability
3. Distributions
4. Moments
5. Estimators
6. Sampling
7. Data types